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DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] Generally, this invention relates to 
the imaging system used by medical diagnosis. This 
invention relates to transmitting a digital image to the 
remote device for carrying out an archive, observation, and/ 
or a print from an ultrasonic imaging system with a network 
in detail. 
[0002] 

[Background of the Invention] In the conventional ultrasonic 
imager, the body tissue and the two-dimensional picture of a 
blood flow are generated by scanning the ultrasonic beam 
converged in a certain scan layer, and detecting the 
ultrasonic energy reflected along each scanning line in a 
scan layer for every transmit beam. Subsequently single 
scanning lines (or scanning line group by which the ** 
office was carried out to the narrow range) are collected by 
receiving time later on in reflection energy by converging 
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ultrasonic energy on one point and transmitting. The B | 

mode ultrasonographic image comprises two or more image | 

scanning lines. The luminosity on the display screen of a | 

certain pixel is based on the intensity of an echo reflected | 

from the body tissue under scan. Coherent addition of the | 

output of a received beam formation machine channel is | 

carried out, and each pixel intensity value over each sample § 

volume for a scan is formed. Logarithmic compression of | 

these pixel intensity values is carried out, and after carrying I 

out scanning conversion, they are displayed as a B mode | 

image of the scanned dissection structure. | 

[0003]If an ultrasonic probe is made to sweep on a part with | 

the body, a series of image frames (it corresponds to the | 

slice which vacated the interval which intersects the body | 

currently inspected) can be displayed on a monitor. In a | 

certain type of ultrasonic imaging system, one long | 

sequence which consists of the newest pictures is | 

memorized in a cinema memory, and is continuously | 
updated automatically in accordance with first-in, first-out. 

This cinema memory is like [ incorporating the image data I 
displayed in real time to a user ] the circulation picture 
buffer which is operating in the background. A cinema 

memory plays the role of the buffer for transmitting a f 
picture to a digital archive device via a host computer. 

When a user makes a system stand it still (operation of the I 

suitable device by an operator interface), the user can I 

observe the image data incorporated into the cinema I 

memory before. In order to be able to examine the picture I 

loop which the cinema memory was made to memorize on a I 
display monitor via the trackball control included in the 

operator interface and to make it memorize on a hard disk, it I 

can choose a section with a picture loop. Arbitrary I 

collection pictures or projection images can be made to | 

memorize on the magneto-optical disc (MOD) inserted in I 

carrying out internal memory on the hard disk of a system, I 

or disk driving. 1 

[0004]Besides memorizing a picture inside, it is required in I 

the latest imaging system to be able to transmit a picture to a | 

remote device various type via a communication network. In | 

order to carry out image transfer correctly, it is required for | 

the network function to which an imager relates to have the | 

network function and compatibility of a remote device of a | 

transmission destination. In detail, the imager needs to | 

arrange the data to transmit in the form that the remote | 

device of a transmission destination can be dealt with. There | 

is standard adoption for DICOM (Digital Imaging and | 
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Communications in Medicine) which specifies the | 

requirements for conformity over a related network function | 

as one of the trials which is going to attain the above- | 

mentioned thing. A standard [ for DICOM ] one aims at | 

using it, when transmitting a medical digital image between | 

a printer, a workstation, collection modules (ultrasonic | 

imaging system etc.), and a file server. This collection | 

module is programmed to transmit data in the form which | 

suited a standard [ for DICOM ] one. | 

On the other hand, the receiving device is programmed to | 

receive the data by which formal setting out was carried out | 

so that it might be based on a standard [ for these same | 

DICOM(s) ] one. | 

[0005]The DICOM system is designed, for example make | 

easy communication of a digital image various type [, such | 

as roentgenography, computerized transaxial tomography, | 
magnetic resonance imaging, and ultrasonic imaging, ]. All 
DICOM operations are processed by a queuing method 

(queued manner) by the application software which operates I 
on the host computer included in the imager. In a certain 
type of ultrasonic imager, the user can choose the arbitrary 

pictures which it is going to transmit to the remote device I 

which has a DICOM function via LAN in DICOM form I 

within a cinema memory. The host computer of this f 

ultrasonic imaging system is programmed with the system I 

software of DICOM, Transmission of the image frame from I 

the cinema memory which passed the hard disk and LAN of I 
the host computer with this software to a remote DICOM 
device becomes easy. 

[0006] Although DICOM has prescribed how to transmit the I 

picture of a single frame and multiframe in the form of a I 

DICOM object covering a long distance via a network, the | 

method of transmitting a live streaming image is not | 

specified. The conventional way stage for transmitting the I 

live image stream of an ultrasonographic image from an 1 

ultrasonic scanner to an observation station is the frame | 

incorporation device (frame-grabbing device) connected to I 

the video output port in the back of an ultrasonic device. | 

This frame incorporation device ranks second and is passing | 

that information to another computer located in a receiver. | 

The method of providing a live streaming image is needed | 

without the necessity of purchasing the special computer | 

which creates a streaming image, by incorporating a | 

continuous frame. | 

[0007] As an exception method, the live streaming image has | 
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been provided by making the RGB output and synthetic | 

video output of an ultrasonic imaging system only extend. In | 

order to realize this, use a special RGB cable and a synthetic | 

video cable, and. Inserting in the analog RGB of a monitor | 

or composite "in" which inserted one such cable side in | 

"out" of the analog RGB on an ultrasonic device or | 

composition, and has arranged another side to the remote | 

observation station requires. In this technique, since it is | 

difficult to be able to perform only problem solving in a | 

short distance (about 100 feet or less), and to deal with two 1 

or more ultrasonic scanners, this technique is not helpful. | 

Only the mechanism of incorporation and record serves as | 

VCR. For this reason, it does not become the solution by | 

digital one. | 

[0008]The ultrasonic imaging system which can transmit a | 

picture to a remote device using a TCP/IP network is | 
indicated by U.S. Pat. No. 5,897,498. This ultrasonic 
imaging system, Via network connection, such as an 
Ethernet port and a modem, via a local network or the 
Internet by TCP/IP. The SMTP (simple mail transfer 
protocol: mail transfer protocol) server which transmits an 

electronic message is included. It is connected to the control I 

device of an ultrasonic system, and this SMTP server can I 

have a dialog with the storage of an ultrasonic system, a user 1 

interface, and a display. The preset value of an I 

ultrasonographic image, a report, an ultrasonographic image I 

loop, and a system, etc. can attach the information I 

memorized on an ultrasonic system to this electronic I 
message, in order to transmit to a receiving station. 
However, formation of such an electronic message is not a 

live streaming image. I 

[0009]Therefore, the method and means which make it I 

possible to transmit a live streaming image to a receiving J 

device directly by a Wide Area Network with an ultrasonic 1 

imaging system are required. | 

[0010] I 

[Means for Solving the ProblemJThis invention is included I 

in an ultrasonic imaging system programmed to have a 1 

function of a live streaming image. Since bandwidth and a | 

processing unit which can attain such a task without | 

reducing an action of an ultrasonic device were needed, it | 

was not thought that it was clear once to include a streaming | 

image in an ultrasonic device. Solution by digital one | 

advocated on these specifications, An image displayed on a | 

monitor of an ultrasonic scanner is captured in a form of a | 

single frame or a frame group, is compressed by MPEG | 
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form, and ranks second, It is transmitted to a receiving set | 

(for example, personal computer linked to a network) via a | 

TCP/IP network through a digitized output port of a scanner. |j 

A host computer of an ultrasonic imaging system, It is | 

programming with live streaming image software which | 

makes easy transmission to a remote receiving set from a | 

cinema memory of an image frame through a hard disk and | 

a network of this host computer. A software application | 

which is operating on a receiving set outputs an image and a I 

sound to a monitor which collects streaming data, and | 

processes this data, and is in a receiver. It is important for a | 

receiving station to make it output smoothly by buffering | 

data which receives a message. According to this desirable | 

embodiment of this invention, an ultrasonic scanner is | 

supporting transmission of a streaming image and a sound | 

through a TCP/IP network, and, thereby, the receiving set | 
can process and output an image and a sound in real time 
mostly. A live streaming image can be transmitted to 

arbitrary receiving devices which are using a TCP/IP | 

protocol for the network. I 
[001 1] According to another mode of this invention, the 
ultrasonic imaging system can broadcast a live streaming 

image in two or more parts. This streaming image attaches a I 
header for identifying two or more IP addresses, and can be 

transmitted to a gateway using a TCP/IP protocol. I 

[0012]This invention is extended to two-way I 

communication between an ultrasonic device and a device in I 

a remote position for observing a live streaming image. This I 

two-way communication contains the capability to carry out I 
communication through either indication of structure found 

out on a picture, writing of a up to [ a picture ], drawing and I 

a voice message or a message of a text. This two-way I 

communication is performed while connection between an I 

ultrasonic imaging apparatus and an observation station is 1 

kept open. | 

[0013] I 

[Embodiment of the Inventionl Drawing 1 illustrates the 1 

conventional computer type ultrasonic imaging system | 

programmable to transmit a live streaming image to a | 

remote device via a TCP/IP network. The imaging system of | 

the type shown in drawing 1 has an B mode which makes | 

the organization which reflects an echo, and/or the two- | 

dimensional picture of a blood flow generate by an imager. | 

The basic chain of signal processing is as follows. | 

[00 14] The array 2 of the ultrasonic transducer received | 

starting with the transmitter in the beam forming machine 4, | 
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and has transmitted the sound burst converged on the point | 

of having met the scanning line. The reflected RF signal is | 

detected by a transducer element, and it is dynamically | 

converged so that a received beam may subsequently be | 

made to form with the receiver in the beam forming | 

machine 4. The output data (I/Q or RF) of the received beam I 

formation machine to each scanning line passes the B mode 1 

processing chain 6. As for this B mode processing chain 6, it | 

is preferred that a recovery, filtering, envelope detection, | 

logarithmic compression, and edge enhancement are | 

included. | 

[00 15] A single sound image frame can be made to form at | 

the maximum using hundreds of receiving vectors according | 
to the geometrical composition of the scan. Since the time 

shift to the following sound frame from a certain sound | 

frame is made smooth, a certain sound frame equalization 8 | 

may be performed before scanning conversion. Generally, in | 
order to carry out graphic display of the indicative data 
which carried out logarithmic compression, it is changed 

into a X-Y format with the scan converter 10. In some f 
systems, to the sound frame before scanning conversion, 
there is nothing then and frame equalization may be 

performed to X-Y data (the block 12 of a dashed line I 
shows). Occasionally, in order to obtain a given graphic 

display frame rate, a copy video frame may be inserted I 

between sound frames. The frame which carried out I 

scanning conversion is sent to the image processor 14 which I 

maps picture image data using gray scale mapping. I 

Subsequently, the image frame used as the gray scale is sent I 
and displayed on the picture monitor 18. 
[0016]In control of a system, an operator input is accepted 
via the operator interface 22, and control is centralized on 

the host computer 20 which is controlling various f 

subsystems to this. ( Drawing 1 shows only the transfer path | 

of image data.) The operator interface is provided with other | 

input devices, such as a keyboard, a trackball, many push I 

buttons and a sliding type knob, and a revolving knob. 1 

[00 17] One long sequence which consists of the newest | 

pictures between imaging is memorized in the cinema | 

memory 16, and it is updated automatically continuously. | 

Some systems are designed save a R-theta sound picture | 

(the dashed line of drawing 1 shows this data path), and save | 

the X-Y video image in another system. In order to be able | 

to examine the picture loop made to memorize in the cinema | 

memory 16 via trackball control and to store it in a hard | 

disk, it can choose a section with a picture loop. | 
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[0018]The ultrasonic imaging system programmed so that | 

drawing 2 outputted the live stream of a video image by | 

compression format via the transmitting TCP/IP network | 

connection 24 as a whole, The observation station | 

programmed to receive the live stream of a video image via | 

the receiving TCP/IP network connection 26, The simple | 

system provided with the network 28 for connecting the | 

transmitting TCP/IP network connection 24 to the receiving | 

TCP/IP network connection 26 is expressed. The network | 

28 can contain arbitrary network systems in addition it | 

contains the system to which interconnection of the network | 

was carried out via a local area network, a Wide Area | 

Network, the intranet in a company, the Internet, and a | 

gateway. The ultrasonic imaging system selectively shown | 

by drawing 2 is a thing of the type in which the whole was | 
shown more in drawing 1 , and the same reference number 
has expressed the component with same function. 
[0019]The frame (.) which the host computer 20 of an 

ultrasonic imaging system becomes from a digital video data I 

according to this desirable embodiment of this invention Or I 

it is programmed to have a function which extracts a frame I 

group from the cinema memory 16, and compresses this I 
video frame by MPEG form. As for the compressed video 

frame, it is preferred to make it memorize on the hard drive 1 
of a host computer. The compressed video frame is 

transmitted to the transmitting TCP/IP network connection I 

24 from a hard drive. I 

[0020] a group which Moving Picture Experts Group I 

(MPEG group) which is one of the workgroups of ISO I 
developed when the term of "MPEG" was used on these 
specifications - a digital video compression standard and a 

file format are pointed out. Generally in MPEG, a more § 

nearly quality image is created compared with the form of 1 

competing. An MPEG file can be decoded with special I 

hardware or software. In MPEG, the high compression ratio I 

is attained by saving only the change on another frame from 1 

a certain frame rather than saving each whole frame. 1 

Subsequently, this video information is encoded using the | 

technique called a discrete cosine transform (DCT). DCT is | 

the technique for expressing a data point as total with | 

dignity of cosine. DCT is used well because of a data | 

compression. MPEG data compression technology is | 

reducing data volume by estimating some DCT coefficients, | 

and since a part of the result data is removed, it serves as | 

irreversible compression. However, as for this loss of data, | 

not understanding by people's eyes is common. | 
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[0021]There are two, MPEG-1 and MPEG-2, as main | 

MPEG standards. In the most general real original form of | 

MPEG-1 standard, the image resolution of 352x240 is | 

obtained in 30-frame per second (fps). Thereby, the image | 

quality which is a little lower than the quality of the | 

conventional VCR image is acquired. MPEG-2 is a new | 

standard rather than providing the resolution of 720x480 and | 

1,280x720 in 60fps with [ of CD quality ] a full sound. By J 

MPEG-2 data-compression technique, the image for 2 hours | 

is compressible into several gigabytes. Although just slight | 

count ability is enough to carry out compression releasing of | 

the data stream of MPEG-2, in order to encode an image in | 

MPEG-2 form compared with this, quite big throughput is | 

needed. | 

[0022]The host computer 20 is programmed to answer the § 

operation of the live streaming image button (not shown) on | 

the operator interface 22, and to output a live streaming | 
image by compression format. After operating a live 
streaming image button, a system operator answers the 

prompt from a host computer, and inputs the IP address of a I 
remote observation station via the operator interface 22. 
Subsequently, the host computer 20 attaches the demand 
which makes connection with this IP address open, and 
transmits the IP address concerned to the transmitting TCP/ 

IP network connection 24. After connection is established, I 

the positive acknowledge (ACK) that connection opened is I 

transmitted to the host computer 20 from the transmitting | 

TCP/IP network connection 24. Subsequently, a host 1 

computer starts transmission to the transmitting TCP/IP I 
network connection 24 of the frame of a compression video 
data. 

[0023]The transmitting TCP/IP network connection 24 An 

output port (for example, Ethernet port), The TCP/IP 1 

software module for making a compression video data 1 

transmit from an output port according to the network | 

protocol software called a TCP/IP Internet Protocol sweet I 

(Suite) is included. TCP/IP is connected with Internet 1 

Protocol (IP) and a data link control protocol (TCP), and is | 

named. IP software controls the route setting of data, and | 

TCP is controlling the data transfer. The TCP/IP software | 

module has confined picture image data in the segment | 

called a TCP packet, a data segment is supervised and | 

checked to this segment, and the header information for | 

making it align in a proper order is attached to it. Since the | 

block of data is transmitted via the Internet in the form of a | 

discrete packet as for which each packet is made to the route | 
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setting which changes with gateways, these packets may | 

include the error, when an order is different in the | 

transmission destination, and it arrives at it or it arrives. In | 

the receiving TCP/IP network connection 26, it is confirmed f) 

whether these packets are errorless according to the header | 

information of a TCP packet, If there is no error in a | 

segment, a positive acknowledge will be taken out, and | 

further, these packets are arranged so that setting up the | 

original block of picture image data may be finished again. | 

The transmitting TCP/IP network connection 24 is | 

monitoring the positive acknowledge of a segment | 

continuously, and when the positive acknowledge of the | 

segment is not carried out to timely, the transmitting TCP/IP | 

network connection 24 resends the packet. When a segment | 

is lost at the time of the first transmission or is received in | 

the wrong order, TCP software, When it continues holding | 

the segment received until all the segments were obtained in | 
the receiving TCP/IP network connection 26 and all are 

obtained, it is proper, a segment is arranged in an order | 
without an omission, and it can refinish setting up the 
original data block. 

[0024] In the transmitting TCP/IP network connection 24, a 

TCP packet receives processing by IP software which is I 

changing the segment to the form of an IP packet. The IP 1 

header which provides the address information used in order I 

that a gateway may carry out route setting to a proper I 

transmission destination to a packet is contained in each I 

packet. The IP header includes the Internet address of the I 

transmitting agency and the transmission destination, and, I 
thereby, the proper route setting of the data based on a 
gateway and the positive acknowledge to the packet 
reception by a receiver of it become possible. Although IP 

software tends to transmit all the packets, the guarantee of J 

these having been transmitted is not obtained by IP I 

software. A transmissive guarantee is obtained as mentioned I 

above by the positive acknowledge and resending by TCP I 

software. 1 

[0025]It is necessary to make TCP/IP software constitute | 

according to a concrete ultrasonic system and its | 

environment. In typical configuration information, by TCP/ | 

IP, the kind of local network (when it is local and network | 

connection of the ultrasonic system is carried out to another | 

ultrasonic device), The address of another system on a local | 

network, a gateway address (when the system is performing | 

the route setting function), The address of the server on the | 

user name of an ultrasonic device and the password for | 
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access, and an ultrasonic system and the Internet address (IP | 

address) to the ultrasonic system are included. | 

[0026] Although it only aimed at the graphic display, The | 

reception observation station by one gestalt (as [ show / in | 

drawing 2 1 ) of desirable operation is provided with the | 

receiving TCP/IP network connection 26, the host computer | 

30, the video memory 32, the operator interface 34, the | 

image processor 36, and the display monitor 38. This | 

receiving TCP/IP network connection 26 is provided with | 

the output port (for example, Ethernet port) and the TCP/IP | 

software module for receiving a compression video data | 

according to a TCP/IP protocol. The receiving stream of a | 

compression video data is taken out from the receiving TCP/ | 

IP network connection 26 by the host computer 30, and is | 

memorized on the hard drive. The host computer 30 is | 

programmed by the software for carrying out compression | 
releasing of the compression video data. The host computer 
30 carries out compression releasing of the picture image 

data, and is outputting the picture image data which carried | 

out compression releasing to the video memory 32 which 1 
carries out the duty of a buffer. Subsequently, the live 
stream of a video image is reconstructed from the picture 
image data stored by the image processor 36, and it is sent 
and displayed on the display monitor 38. The doctor in 
charge who is present in a remote observation station can 

observe the live streaming image which an ultrasonic I 

imaging system outputs. I 

[0027]When using it on these specifications, the term of I 

"streaming" means the technique for transmitting data so I 

that processing may become possible as a stream which was I 

steady and continued. In order to operate streaming well, the I 

host computer 30 of a receiver performs collection to picture | 

image data, compression releasing, and transmission as a 1 

regular stream to the video memory 32 for making it process 1 

by the image processor 36. The video memory 32 works as a 1 

buffer and the picture image data exceeded when the I 

receiving station had received picture image data at the 1 

speed exceeding a demand is stored. | 

[0028] According to this desirable embodiment of this | 

invention, the host computer 20 of an ultrasonic imaging | 

system transmits a message to the transmitting TCP/IP | 

network connection 24, and it is ordering it it so that | 

connection with the transmission destination device which | 

the host computer specified may be made to open. When the | 

transmitting TCP/IP network connection 24 is connected to | 

the receiving TCP/IP network connection 26 of an | 
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observation station via the network 28, this connection, It is | 
maintained by the open state while the host computer 20 | 
supplies the frame of a compression video data via this | 
connection. Each of a frame is a still picture. Compression | 
releasing of these frames is carried out by the host computer | 
30, and they continue on the display monitor 38 by the | 
image processor 36, and are displayed. An illusion that it is fj 
moving is made to have by displaying a frame continuously 
at high speed. The motion displayed becomes smoother, so | 
that there are many frame numbers per second. According to f 
which MPEG data compression technique is used, the frame | 
rate by this desirable embodiment becomes 60 either 30fps 
or fps. Generally, in order to keep a motion from being 
awkward, about 30 fps(es) are required at the minimum. 
[0029]Although it was made to relate to a desirable 
embodiment and this invention has been indicated, he can 
understand that various change can be made and it can 
substitute for the component with an equivalent, without 
deviating from the range of this invention, if it is a person 
skilled in the art. Many corrections which adapt the concrete \ 
situation which exists without deviating from the essential 
range of this invention to instruction of this invention are 
possible. For example, live streaming picture image data can I 
be transmitted not via an Ethernet port but via a modem. 
Therefore, this invention is not the thing intended so that it 
might limit to the concrete embodiment indicated as the 
optimal mode planned so that this invention might be 
carried out, and this invention is meant so that all the 
embodiments belonging to this claim may be included. 
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